As the title is self-descriptive, I need to plot the training and validation accuracy obtained during the training of my Hugging Face model. After that, I'd like to plot the confusion matrix for the test predictions. How can I do these?
Here is my training arguments:
args = TrainingArguments(
output_dir=f"my_training",
evaluation_strategy="epoch",
save_strategy="epoch",
learning_rate=5e-5,
per_device_train_batch_size=4,
gradient_accumulation_steps=4,
per_device_eval_batch_size=4,
num_train_epochs=5,
warmup_ratio=0.1,
logging_steps=10,
load_best_model_at_end=True,
metric_for_best_model="accuracy",
report_to='tensorboard',
push_to_hub=True,
)
And, here is my trainer:
def compute_metrics(eval_pred):
predictions = np.argmax(eval_pred.predictions, axis=1)
accuracy = accuracy_score(y_pred=predictions, y_true=eval_pred.label_ids)
return {"accuracy": accuracy}
trainer = Trainer(
model,
args,
train_dataset=train_dataset,
eval_dataset=eval_dataset,
tokenizer=processor,
compute_metrics=compute_metrics,
data_collator=collate_fn
)
Finally, I start the training and prediction, respectively:
train_results = trainer.train()
trainer.save_model()
trainer.log_metrics("train", train_results.metrics)
trainer.save_metrics("train", train_results.metrics)
trainer.save_state()
eval_results = trainer.evaluate(eval_dataset)
trainer.log_metrics("eval", eval_results)
trainer.save_metrics("eval", eval_results)
With the current configuration, I only get the eval/accuracy vs. Steps graph. I need a plot like the one given below, which was taken from TensorBoard:
https://preview.redd.it/pzhk797r7dfc1.jpg?width=478&format=pjpg&auto=webp&s=955a22ef695a8945d2faf0dd8155329535834a8b
submitted by /u/talhak
[link] [comments]